Discovering Closed-Loop Failures of Vision-Based Controllers via Reachability Analysis

Machine learning driven image-based controllers allow robotic systems to take intelligent actions based on the visual feedback from their environment. Understanding when these might lead system safety violations is important for integration in safety-critical applications and engineering corrective measures system. Existing methods leverage simulation-based testing (or falsification) find failures of vision-based controllers, i.e., inputs that closed-loop violations. However, techniques do not scale well scenarios involving high-dimensional complex inputs, such as RGB images. In this work, we cast problem finding vision a Hamilton-Jacobi (HJ) reachability problem. Our approach blends analysis with HJ compute an approximation backward reachable tube (BRT) system, set unsafe states under controllers. Utilizing BRT, can tractably systematically corresponding failures. These be subsequently analyzed input characteristics have caused failure. Besides its scalability explicit computation BRT allows proposed capture non-trivial are difficult expose via random simulations. We demonstrate our framework two case studies neural network controller (a) autonomous indoor navigation, (b) aircraft taxiing.